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Wc use local polynomial fitting to estimate the nonparametric M-regression function for strongly mixing 
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statistical inference and for applications that involve plugging in such estimators into other functionals 

' where some control over higher order terms are required. We apply our results to the estimation of an 
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. additive M-regression model. 
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1 Introduction 

In many contexts one wants to evaluate the properties of some procedure that is a functional of 
some given estimators. It is useful to be able to work with some plausible high level assumptions 
about those estimators rather than to rederive their properties for each different application. 
In a fully parametric (and stationary, weakly dependent data) context it is quite common to 
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assume that estimators are root-n consistent and asymptotically normal. In some cases this 
property suffices; in other cases one needs to be more explicit in terms of the linear expansion 
of these estimators, but in any case such expansions are quite natural and widely applicable. 
In a nonparametric context there is less agreement about the use of such expansions and one 
often sees standard properties of standard estimators derived anew for a different purpose. 
It is our objective to provide results that can circumvent this. The types of application we 
have in mind are estimation of semiparametric models where the parameters of interest are 
explicit or implicit functionals of nonparametric regression functions and their derivatives; see 
Powell (1994), Andrews (1994) and Chen, Linton and Van Keilegom (2003). Another class of 
applications includes estimation of structured nonparametric models like additive models (Linton 
and Nielsen, 1995) or generalized additive models (Linton, Sperlich and Van Keilegom, 2007). 

We motivate our results in a simple i.i.d. setting. Suppose we have a random sample 
{Yi, Xi}"^^ and consider the Nadaraya- Watson estimator of the regression function m{x) = 
E{Y^\Xi = x), 

-i ^ _ r{x) _ n-^ ELi KhjXj - x)Yi 
""^"^^ /(x) n^'T.UKH{X^-x)' 

where K \s a, symmetric density function, /i is a bandwidth and Kh{.) = K{./h)/h. Standard 
arguments (Hardle, 1990) show that under suitable smoothness conditions 

1 " 

rh{x) - m(x) = h^b{x) + „, , Kh{Xi - x)ei + Rn{x), (1) 

where b{x) = J v?K{u)du{m" ix) + 2m' (x) f (x) / f {x)]/2, while f{x) is the covariate density and 
Ei = Yi — m{Xi) is the error term. The remainder term Rn{x) is of smaller order (almost surely) 
than the two leading terms. Such an expansion is sufficient to derive the central limit theorem 
for m(x) itself, but generally is not sufficient if rh{x) is to be plugged into some semiparametric 
procedure. For example, suppose we estimate the parameter Oq = f m{x)'^dx hj 6 = f m[x)^dx, 
where the integral is over some compact set V; we would expect to find that ni/2(^ - ^o) is 
asymptotically normal. Based on expansion ([1]), the argument goes like this. First, we obtain 
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the following 

n^l'^[Q — ^q) = 2in}^'^ j •m{x){rh{x) — m{x)}dx + n^^"^ J [m{x) — m{x)]'^dx. 

If it can be shown that m{x) — m{x) = o{n~^/^) a.s. uniformly in x € 2? ( such results are widely 
available; see for example Masry (1996)), we have 

n^^'^{9 — 6q) = 2n^^'^ J m{x){'m{x) — m{x)}dx + o{l), a.s. 

Note that the quantity on the right hand side is the term in assumption 2.6 of Chen, Linton, 
and Van Keilegom (2003) which is assumed to be asymptotically normal. It is the verification 
of this condition with which we are now concerned. If we substitute in the expansion ([T]) we 
obtain 

n^/\e -eo) =2n^/^h^ [ m{x)b{x)dx + 2n^/^ [ "^^^n^^Y Kh{X, - x)eidx 

J J f{^) ^ 

+ 2n^/^y m{x)Rnix)dx + o{l), a.s. 

If nh^ — > 0, then the first term (the smoothing bias term) is o(l). By a change of variable, the 
second term (the stochastic term) can be written as a sum of independent random variables 
with zero mean 

n n 

nV2 J m(x)/-i(x)n-i Yl Kh{Xi - x)e^dx = n-V2 ^ ^„(Xi)ei, 

i=l i=l 

UXi) = J m{X, + uh)r\Xi + uh)K{u)du, 

and this term obeys the Lindeberg central limit theorem under standard conditions. The problem 
is that equation ([1]) only guarantees that J m{x)Rn{x)dx = o(n~2/5) a.s. at best. Actually, in 
this case it is possible to derive a more useful Bahadur expansion (Bahadur, 1966) for the kernel 
estimator 

n 

m{x) - mix) = h^hn{x) + {Ef{x)y^n~^ ^ Kh{Xi - x)£^ + Rl{x), (2) 

i=l 

where hn{x) is deterministic and satisfies hn{x) b{x) uniformly m x (z T>, and Ef{x) f{x) 
uniformly in x € P, while the remainder term now satisfies 

sup|K(x)j = ofi^') a.s. (3) 

xGD \ nh J 



This property is a consequence of the uniform convergence rate of /(x) — Ef(x), n ^ J2^=i Kh{x 
-Xi){m{X,) - m{x)} - EKh{Xi - x){m{Xi) - m{x)} and YJLi Kh{X, - x)ei that fohow 
from, for example Masry (1996). Clearly, by appropriate choice of /i, Rn{x) can be made to 
be o(n~^/^) a.s. uniformly over V and thus 2?i^/^ J m{x)R^{x)dx = o(l) a.s.. Therefore, to 
derive asymptotic normality for n^^^{9 — 6q), one can just work with the two leading terms 
in ([2]). These terms are slightly more complicated than in the previous expansion but are still 
sufficiently simple for many purposes; in particular, bn{x) is uniformly bounded so that provided 
nh^ 0, the smoothing bias term satisfies h'^n^^^ J m(x)bn{x)dx — > 0, while the stochastic term 
is a sum of zero mean independent random variables 



/, s n n 



/ -.^ N ( fniXi + uK) , , 

= / ) ' ' K{u)du, 

J f{Xi + uh) 

and obeys the Lindeberg central limit theorem under standard conditions, where f{x) = Ef{x). 
This argument shows the utility of the Bahadur expansion ([2]). There are many other applica- 
tions of this result because a host of probabilistic results are available for random variables like 
X]r=i ^h{Xi — x)ei and integrals thereof. 

The one-dimensional Nadaraya- Watson estimator for i.i.d. data is particularly easy to an- 
alyze and the above arguments are well known. However, the limitations of this estimator are 
manyfold and there are good theoretical reasons for working instead with the local polynomial 
class of estimators (Fan and Gijbels, 1996). In addition, for many data especially financial time 
series data one may have concerns about heavy tails or outliers that point in the direction of 
using robust estimators like the local median or local quantile method, perhaps combined with 
local polynomial fitting. We examine a general class of (nonlinear) M-regression function (that 
is, location functionals defined through minimization of a general objective function p{ )) and 
derivative estimators. We treat a general time series setting where the multivariate data are 
strongly mixing. Under mild conditions, we establish a uniform strong Bahadur expansion like 
([2]) and ^ with remainder term of order (log n/n/i'^)^/^ almost surely, which is almost optimal 



or in other words can't be improved further based on the results in Kiefer (1967) under i.i.d. 
setting. The leading terms are linear and functionals of them can be analyzed simply. The 
remainder term can be made to be o(n~^/^) a.s. under restrictions on the dimensionality in 
relation to the amount of smoothness possessed by the M-regression function. 

The best convergence rate of unrestricted nonparametric estimators strongly depends on 
d, the dimension of x. The rate decreases dramatically as d increases (Stone, 1982). This 
phenomenon is the so-called "curse of dimensionality". One approach to reduce the curse is by 
imposing model structure. A popular model structure is the additive model assuming that 

m(xi, ... ,Xd) =c + mi{xi) + ... + md{xd), (4) 

where c is an unknown constant and ■mi.{.), k = 1, . . . ,d are unknown functions which have been 
normalized such that Errn^[yik) = for k = 1, . . . , d. In this case, the optimal rate of convergence 
is the same as in univariate nonparametric regression (Stone, 1986). An additive M-regression 
function is given by @ with m{x) being the M-regression function defined in Previous 
work on additive quantile regression, for example, includes Linton (2001) and Horowitz and Lee 
(2005) for the i.i.d. case. An interesting application of the additive M-regression model is to 
combine ^ with the volatility model 

Yi = aiSi and Incr^^ = m(Xj), 

where = (yi_i, Y^-rf)'''. We suppose that satisfies ^^[(^(Ine?; 0)|Xj] = for some function 
99(.), whence m{.) is the conditional M-regression of InK^^ given Xi. Peng and Yao (2003) 
have applied LAD estimation to parametric ARCH and GARCH models and have shown the 
superior robustness property of this procedure over Gaussian QMLE with regard to heavy tailed 
innovations. This heavy tail issue also arises in nonparametric regression models, which is why 
our procedures may be useful. Empirical evidence also suggest that moderately high frequency 
financial data are often heavy tailed. We apply our Bahadur expansions to the study of marginal 
integration estimators (Linton and Nielsen, 1995) of the component functions in additive M- 
regression model in which case we only need the remainder term to be o(n~*'/(^P+^)) a.s., where 
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p is a smoothness index. 

Bahadur representations (Bahadur, 1966) have been widely studied and apphed, with notable 
refinements in the i.i.d. setting by Kiefer (1967). A recent paper of Wu (2005) extends these 
results to a general class of dependent processes and provides a review. The closest paper to 
ours is Hong (2003) who established a Bahadur representation for essentially the same local 
polynomial M-regression estimator as ours. However, his results are: (a) pointwise, i.e., for a 
single X only; (b) the covariates are univariate; (c) for i.i.d. data. Clearly, this limits the range 
of applicability of his results, and specifically, the applications to semiparametric or additive 
models are perforce precluded. 

2 The General Setting 

Let {iYi,X_^} be a jointly stationary processes, where = (xji, ...,Xjrf)''' with d> 1 and Yi is 
a scalar. As dependent observations are considered in this paper, we introduce here the mixing 
coefficient. Let F* be the a— algebra of events generated by random variables {{Yi,X_j)^ s < i < 
t}. A stationary stochastic processes {{Yi,X_^)} is strongly mixing if 

sup \P[AB] - P[A]P[B]\ = 7[/c] ^0, as A; ^ oo, 

and 7 [A;] is called the strong mixing coefficient. 

Suppose p(.; .) is a loss function. Our first goal is to estimate the multivariate M-regression 
function 

m{xi, ■■■ ,Xd) = argmin E{p(Yi;e)\X_i = (xi, • • • ,Xd)}, (5) 
6 

and its partial derivatives based on observations {(i^,^j)}iLi- An important example of the 
M- function is the q—th {0 < q < 1) quantile of Yi given = {xi, ■ ■ ■ ,Xrf)''', with loss function 
given by p{y; 6) = {2q — l)(y — 6) + \y — 6\. Another example is the Lq criterion p{y; 9) = \y — 9\'^ 
for q > 1, which includes the least square criterion p{y\9) = {y — 9)"^ with m{.) the conditional 
expectation of Yi given X_^. Yet another example is the celebrated Huber's function (Huber, 
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1973) 

p{t) = t'^/2I{\t\ <k} + {k\t\ - k'^/2)I{\t\ > k}. (6) 

Suppose m{x) is differentiable up to order p + 1 at x = (xi, x^)"''. Then the multivariate p'th 
order local polynomial approximation of m(z) for any z close to x is given by 

m(z) = —;D-m(x)(z — x)-, 

0<|r|<p ~' 

where r = (ri, r^), |r| = X^f^j^ rj, r! = ri! x • • • x r^! and 

1 <^ 0<|r|<p j=Or-i=0 rd=0 

ri + ...+r-d=i 

Let i^(u) be a density function on B!^ ^ h a bandwidth and Kfi{u) = K{u/h). With observations 
{(li, we consider minimizing the following quantity with respect to < |r| < p 

n 

Y.mx,-x)p(Yv, (8) 

i=l 0<|r:|<p 

Denote by (ir{x), < \r\ < p, the minima of ([8]). The M- function m{x) and its derivatives 
D-m{x) are then estimated respectively by 

iTT'{x) = l3o{x) and D-m{x) = riPr{x), 1 < |r| <p. (9) 

3 Main Results 

In Theorem 3.2 below we give our main result, the uniform strong Bahadur representation for 
the vector j3p{x). We first need to develop some notations to define the leading terms in the 
expansion. 

Let Ni = be the number of distinct d— tuples r with |r| = i. Arrange these d— tuples 

as a sequence in a lexicographical order (with the highest priority given to the last position so 
that (0, • • • ,0,i) is the first element in the sequence and («,0, ••• ,0) the last element). Let 
Ti denote this 1-to-l mapping, i.e. Ti(l) = (0, • • • ,0,i),-- - ,Ti{Ni) = (i,0, ••• ,0). For each 



7 



i = 1, - ■ ■ ,p, define a A'^j x 1 vector fii{x) with its kth element given by and write /i(^) = 

(1, • • • ■,fJ'p{xyy, which is a column vector of length N = Yl^i=0'^i- Similarly define 
vectors j3p{x) and (3 through the same lexicographical arrangement of D-m{x) and (3r in ^ for 
< lul < P- Thus ([8j) can be rewritten as 



(10) 



i=l 



Suppose the minimizer of (fTOl) is denoted as f3n{x). Let f3p{x) = Wpl3n{x), where Wp is the 
diagonal matrix with diagonal entries the lexicographical arrangement of r!, < |r| < p. 
Let Vi = J K{u)u-du. For g{.) given in (f2^ . define 

i^ni{x) = / K{u)y^g{x_ + hu)f{x + hu)du. 



For < j,k < p, let S'^^/j and Snj,kix) be two A'^- x A^/^ matrices with their {l,m) elements 
respectively given by 



5, 



ill) 



Now define the N x N matrices Sp and Sn,p{x) by 

^n.pix^ — 



Sn 



5*0,0 


5*0,1 ■ 


5*0,^ 


5*1,0 


•5i,i • 


5'i,p 


5p,o 


5'p,i • 


5p^p 



5*n,o,ofe) 5*„,o,l(^) 

5*n,l,o(^) 5*n,l,l(x) 



5'n,p,o(^) 5n,p,l(^) 



5'n,0,p(-£) 
5'n,l,p(^) 

Sn,p,p (^) 



According to Lemma 15.81 Sn^p{x) converges to g{x)f{x)Sp uniformly in x G P almost surely. 
Hence for |5p| 7^ 0, we can define 



1 " 
PUx) = -^WpS-l,{x)H-' J2 Kh{X^ - liMY^^^^{X^ - xfPp{x))fi{X, - x) 



(12) 



i=l 



where ¥?(.;.) is the piecewise derivative of p(.,.), as defined in (Al) and Hn is the diagonal 
matrix with diagonal entries /i'-', < |r | < p in the aforementioned lexicographical order. The 
quantity /3^(x) is the leading term of our expansion; it contains both a bias term, i?/3*(x), and 
a stochastic leading term /3j^(x) — EP*(x). 
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Denote the typical element of f3n{x) by f3nr{x), < |r| < p and the density function of X by 
/(.). The following results on EP*^{x) is an extension of Proposition 2.2 in Hong (2003) to the 
multivariate case. 

Proposition 3.1 If f{x) > and conditions (Al)-(A5) in the Appendix hold, then 

' -hP+^eN{r)WpS-^Bimp+i{x) + o{hP+^), for p - |r| odd, 



Ep:,ix) 



{fg}-~Hx)uip+i{x){M{x) - NpS-^Bi} + B2mp+2{x) 



+o(/iP"'"^), for p — \r\ even, 

where N{r) = T^.^{r) + Yl'^^Lt^ ^k, is a N x 1 vector having 1 as the ith entry with all other 
entries 0, and Bi = [5o,p+i, Si^p+i, • • • Sp^pj^i^ , B2 = [5'o,p+2, 5'i,p+2, • • • Sp^p+2f' ■ 

We next present our main result, the Bahadur representation for local polynomial estimates 
Pp{x). 

Theorem 3.2 Suppose (Al)-(A7) in the Appendix hold with A2 = (p+ l)/2[p + s + 1) for some 
s > and V is any compact subset of R'^. Then 

sup \Hn0p{x) - (3p{x)} - /3*(x)| = ^ ^) ^^'^ost surely, 

where \.\ is taken to he the sup norm and 

w ^ . f P+l 3P+3 + 2S-. 
A(s) = mm <^ -, ->. 

Remark 1. According to Theorem 1 in Kiefer (1967), the point- wise sharpest bound of the 
remainder term in Bahadur representation of the sample quantiles is (log log n/n)^/^. As A(0) = 
3/4, we could safely claim the results here could not be further improved for a general class 
of loss functions p{.) specified by (Al) and (A2). Nevertheless, it is possible to derive stronger 
results, if the concerned loss functions enjoy higher degree of smoothness; see ([3]) in which case 
p(.) is the squared loss function. More specifically, suppose ip{.) is Lipschitz continuous and 
(Al)-(A7) in the Appendix hold with A2 = 1/2 and Ai = 1. Then we prove in the Appendix 
that with probability 1 and uniformly in x G V, 

( log n \ 

sup \Bn{iip{x) - iip{x)) - /3*(x)| = O —-r- almost surely. (13) 



Remark 2. The dependence among the observations doesn't have any impact on the rate 
of uniform convergence, given that the degree of the dependence, as measured by the mixing 
coefficient 7[A;], is weak enough such that (|2Up and (j21|) are satisfied. This is in accordance 
with the results in Masry (1996), where he proved that for local polynomial estimator of the 
conditional mean function, the uniform convergence rate is (n/i'^/ log the same as in the 

independent case. 

Remark 3. It is of practical interest to provide an explicit rate of decay for the strong mixing 
coefficient ^[k] of the form ^[k] = 0{\/k'^) for some c > (to be determined) for Theorem 13.21 to 
hold. It is easily seen that, among all the conditions imposed on 7 [A;], the summability condition 
(1211) is the most restrictive. We assume that 



h = hn ^ {logn/ny for some — < a < -ll 



2{p + s + l) + d- dl (l-A2)i^2-4Ai+2(l + A2)J 

whence (jl9p is satisfied. Algebraic calculations show that the summability condition ()2ip is 

satisfied provided that 

(1 - ad){{l - A2)(4iV + 1) + 8N\i} + 10 + (4 + 8N)ad - ^ ^^ ni^ 

'^>^2^T7T ^--7z , r-iwT r ^^-^^ 1 = c((i, p, z^2 , a, Ai , A2 . (14) 

2(1 — A2)(l — ad)i'2 — Sad + 4(1 — ad){l — A2 — 2Ai) 

Note that we would need the following condition 

A{dd+{l-dd)Xi} 
' (l-ad)(l-A2) 

to secure positive denominator for (jl4p . As c{d,p, 1/2, d, Ai, A2) is decreasing in z^2(^ i^i), there is 

a tradeoff between the order vi of the moment E\(p{ei)\'^^ < 00 and the decay rate of the strong 

mixing coefficient 'y[k]: the existence of higher order moments allows 'y[k] to decay more slowly. 

Remark 4. It is trivial to generalize the result in Theorem 3.2 to functionals of the M-estimates 

/3p{x). Denote the typical elements of Pp{x) and (3p{x) by Pprix) and Pprix), < |r| < p 

respectively. Suppose G{.) : R satisfies that for any compact set J) C R'^, there exists 

some constant C > 0, such that \G\ (/3pr(^))| < C and \G" {f)pr{x))\ < C for all x £ V. Then 

with probability 1, 

logni M-i)-^ 



sup 



h\r.\ [G0pr{x)} - G{Ppr{x)] - G'{l3pr{x)}PU2D\ = o{{^} ' ) (15) 
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uniformly for all rr € V. 

The following proposition follows from Theorem 13.21 and uniform convergence of sum of 
weakly dependent zero mean random variables. 

Corollary 3.3 Suppose conditions in Theorem. \3.S\ hold with s = 0. Then with probability 1 we 
have, uniformly in x£ V, 

i=l 

4 M-Estimation of the Additive model 

In this section, we apply our main result to derive the properties of a class of estimators in the 
additive M-regression model In terms of estimating the component functions m}^[.), k = 
1, . . . , d in ([U, the marginal integration method (Linton and Nielsen, 1995) is known to achieve 
the optimal rate under certain conditions. This involves estimating first the unrestricted M- 
regression function m(.) and then integrating it over some directions. Partition = (xi, . . . , Xd) 
as = {xii,X_2i), where xij is the one dimensional direction of interest and X_2j is a d — 1 
dimensional nuisance direction. Let x = {xi,X2) and define the functional 

(pi{xi) = j m{xi,X2)f2{x2)dx2, (16) 

where f2{x2) is the joint density of X_2i- Under the additive structure ([1]), 0i(.) is mi{.) up to 
a constant. Replace m(.) in (fTUj) with (3q{xi,X2) = f3o{x) given by ([9]) and ipiixi) can thus be 
estimated by the sample version of p^ : 

n 

(t>ni{xi) = n^^^f3o{xi,X2i)- 

i=l 

As noted by Linton and Hardle (1996) and Hengartner and Sperlich (2005), cautious choice 
of the bandwidth is crucial for 0„i(.) to be asymptotically normal. They suggested different 
bandwidths be engaged for the direction of interest Xi and the d — \ dimensional nuisance 
direction X_2-, say hi and h respectively. Sperlich et al (1998) and Linton et al (1999) provide 
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an extensive study of the small sample properties of marginal integration estimators, including 
an evaluation of bandwidth choice. 

The following corollary is about the asymptotic properties of 0ni(-)- 

Corollary 4.1 Suppose the support of X_ is [0,1]®"' with strictly positive density function. Let 
the conditions in Proposition \3.3\ hold with T„ = {r{n)/ mm{hi,h)}'^ and the h'^ replaced by 
/ii/i*^^^ in all the notations defined either in ^8\) or /il9\) . If hi oc n~^^^'^'P'^'^\ h = 0{hi) and 
/ fTPj) is modified as 

n/ii/i3(«'-i)/log3n ^ oo, n-^{r{n)Y^/'^dn\ogn/M^n^ oo. (17) 

Then we have 

(n/ii)i/2{0„,(^,) _ h N{eiWpS-^BiErryp+i{xi,X2),a''{xi)), 

where '-^ ' stands for convergence in distribution, 

~^\xi) = I / {fg^r\xi,X2)fi{X2)a\xi,X2)dX2]eiS^^K2KlS^^e[, 

'^^fe) = E[v'^{^)\X_ = x\ and K2 = J^q i^ad KillJfJ-illjdv. In particular for additive quantile 
regression, i.e. p{y; 9) = {2q — — 9) + \y — 9\, we have 

d\xi) = q{l - q){ [ f-\xi,X2)f-H0\xi,X2)fiiX2)dxAe,S;'K2KjS;'e[. 
^ J[0,l]®'*-i 

Remark 5. For conditions in Corollary 14.11 to hold, we would need 3d < 2p + 5, i.e. the order 
of local polynomial approximation increases as the dimension of the covariates X increases. See 
also the discussion in Hengartner and Sperlich (2005). 

Remark 6. Besides asymptotic normality, we could also by applying Theorem 13.21 develop 
Bahadur representations for (pniixi), like those assumed in Linton, Sperlich and Van Keilegom 
(2007). Based on (jlSp . similar results are also applicable to the generalized additive M-regression 
model where G(m(xi, . . . , x^)) = c+mi(xi) + . . .+mrf(xd) for some known smooth function G(.), 
in which case the marginal integration estimator is given by the sample average of G(m(xi , Xr,-)). 
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5 Concluding Remarks 

Our results can be useful in a variety of contexts including estimation of quite general nonlinear 
functionals of M-regression functions, and we have shown in one specific application how they 
can be applied. 

Appendix: Regularity Conditions and Proofs 

For any M > 2, A2 G (0, 1) and Ai G (A2, (1 + M)/^ define 

d„ = {nh'^/\ogny^^^+^^''^\nh'^\ognY''^, r{n) = {nh'^ / log n)^^~^^^''^ , (18) 
M^^) = M{nh'^/\ogn)-^\ M^^ = / ^ {nh'^ / log n)-^\ T„ = {r{n)/hY 

and Ln as the smallest integer such that log n(M/2)^"''""'^ > nMn^ /dn- Let ||.|| denote the 
Euclidean norm and C be a generic constant, which may have different values at each appearance. 
Let £i = Yi — m(Xj) and assume that the following conditions hold. 

(Al) For each y G 7^, p{y',0) is absolutely continuous in 9, i.e., there is a function {p{y;6) = 
(p{y — 9) such that for any 9 TZ, p{y; 9) = p{y; 0) + /^^ ip{y; t)dt. The probability density 
function of is bounded, E{ip{ei)\X_j} = almost surely and E\i.p{ei)\'^'^ < 00 for some 
ui > 2. 

(A2) {p{.) satisfies the Lipschitz condition in {aj, flj+i), j = 0, • • • ,m, where ai < • • • < are 
the finite number of jump discontinuity points of oq = "Oo and 0^+1 = +00. 

(A3) K{.) has a compact support, say [—1, 1]'^'^ and \Hj{u) — Hj{v)\ < C\\u — v\\ for all j with 
< IjI < 2p + 1, where Hj{u) = viK{u). 

(A4) The probability density function of X, /(.) is bounded and with bounded first order 
derivatives. The joint probability density of {X_q,X_i) satisfies f{u,v;l) < C < 00 for all 
/ > 1. 

(A5) For r with \r\ = p + 1, D-m{x) is bounded with bounded first order derivative. 
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(A6) The bandwidth /i — > with 

nh'^/logn^ oo, nh'^+^P+^^^^y logn < oo, n-'^{r{n)y^/'^dnlogn/MP oo, (19) 

for some 2 < z^2 < i^i and the processes {{Yi,2Li)} strongly mixing with mixing 
coefficient 7 [A:] satisfying 
00 

^r{7[fc]}^-2/''2 <ooforsomea> (p + d+l)(l-2/i/2)/c^. (20) 

k=l 

Moreover, the bandwidth h and 7 [A;] should jointly satisfy the following condition 

s/2 rM^^^ 1/2 7[r(n)(2'^2/2/M)2LnMi 
^ I d„ / r(n)(2'^2/2/M)2L"/^2 I / > V ; 



n=l 



(A7) The conditional density fx\Y of X given Y exists and is bounded. The conditional density 
fiKi,Ki+i)\{yuyi+i) (^I'^z+i) given (Yi,y/+i) exists and is bounded, for ah I > 1. 

Remark 7. Assumptions on ip(.) in (Al) and (A2) are satisfied in almost all known robust and 
likelihood type regressions. For example, in gt/i— quantile regression, we have ip{t) = 2ql{t > 
0} + {2q — 2)I{t < 0}, while for the Huber's function ([6]), its piecewise derivative is given by 

(p{t) = tl{\t\ <k} + sign{t)kl{\t\ > k}. 

Note that the condition E{Lp{ei)\2£.i} = a.e. is needed for model specification. Moreover, if 
the conditional density f{y\x) of Y given X is also continuously differentiable with respect to 
y, then as proved in Hong (2003) there is a constant C > 0, such that for all small t and x, 



E 



\^ip{Y-t + a)- ^{Y;a)^ \X = u <C\t\ (22) 
holds for all (a,u) in a neighborhood of {m{x),x). Define 

G{t,u) = E{ip{Y-t)\X = u], Gi{t,u) = {d'/df)G{t,u), i = 1,2, (23) 
then it holds that 

g{x) = Gi{m{x), x) > C > 0, G2{t,x) bounded for all x^ V and t near m(x). (24) 

14 



Assumptions (A3)-(A7) are standard for nonparametric smoothing in multivariate time series 
analysis, see Masry (1996). For example, condition (j20p is needed to bound the covariance of 
partial sums of time series as in Lemma 1 5. 5 1 while (|2ip plays a similar role as (4.7b) in Masry 
(1996). It guarantees that the dependence of the time series is weakly enough such that the 
difference caused by the approximation of dependent random variables by independent ones 
(through Bradley's strong approximation theorem) is negligible; see Lemma [5.41 Of course, (j2ip 
is more stringent than (4.7b) in Masry (1996), which is due to the fact that the loss function 
p{.) considered here is more general than the straightforward square loss. 
Proof of Proposition Write P^{x) = -WpS'^pix) ^l^Li ^m(x)/n, where 

Zm{x) = ^n'/i"''^/.(^i -2l)^(l"i,/x(X, -x)^/3p(rE))/i(X, -x). 

We first focus on EZni{x). Based on (|23l) and (fM|l . we have 

EW{Y,,f,{X,-xj'(3p{x))\X,} = Gif,iX,-xyf3p{x),X,) 

= -g{Xi){miX^-f,iX,-x)'(3pix)} 

+G2{^^{x),X^{m{X,) - fi{X, - xfPpix)y/2 

for some ^i(x) between fJ.{X_^ — x)^ (3p{x) and m{X_^). Apparently, if A^j = x + hy_, then 

m{X;)-^^{X,-xfPp{x) = h^^' Y: ^^y^ + h^^' ^^vj^ + oih^^'). 

lfe|=p+l -■ |fc|=P+2 

Therefore, 

EZm{x) = hP+' f K{v)fg{x + hv)fi{v) ^ ^~'^^\ ^dv 

\k\=p+i 

+hP+^ f K{v)fg{x + hv)f-i{v) Y ^~'^^~\ -dv + o{HP+^) 

\k\=p+2 

= T1+T2. 

Now arrange the Np^i elements of the derivatives D-m{x)/r\ for |r| = p + 1 as a column vector 
mp+i(x) using the lexicographical order introduced earlier and define mp^2{x) in the similar 
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way. Let the x Np^i matrix Bni and the x Np^2 matrix Bn2 be defined as 









Snfi,p+2{x) 


Bnlix) = 


Sn,l,p+l{x) 


, Bn2{x) = 


Sn,l,p+2{x) 




Sn,p,p+l{xL) 




Sn,p,p+2{x±) 



where Sn,i,p+i{x) and Sn,i,p+2{x) is as given by ([TT]) . Therefore, Ti = /i^"'"'^i?„i(x)mp+i(x), 
T2 = hP+'^Bn2ix)inp+2ix), and 

EPnii^) = -WphP+^S-}p{x)Bniix)rnp+^ix) - WphP+^S-l,{x)Bn2{x)uip+2{x) + oihP+^). 

Let Cj, i = 1, ■ ■ ■ ,d he the d x 1 vector having 1 in the ith entry and all other entries 0. For 
< j < p, < A; < p + 1, let Nj^k{x) be the Nj x N]^ matrix with its (Z, m) element given by 



Lm 



i=l 



-j(l)+Tu{m)+e^. 



(25) 



and use these Nj^k{x) to construct a N x N matrix Np{x) and a x Np^i matrix M(x) via 



Np{x) 



No,o{x) No.iix) 
Ni,o{x) A^i,i(x) 



Ni,pi^) 



M{x) 



No,p+iix) 
iVi,p+i(x) 

iVp,p+i(x) 



Np,o{x) Np^iix) ■■■ Np^p{x) 
Then Sn,p{x) = {fg}{x)Sp + hNp{x) + ©(/i^), Bni{x) = {fg}{x)Bi + hM{x) + 0{h'') and 
i?n2(x) = {fg}{x)B2 + 0{h). As S-l{x) = {fgr\x)S-^ - h{fgr\x)Sp^Np{x)Sp^ + 0{h^), 
we have 

-S/3;(x) =WphP+^[{fgr\x)S;^ - h{fgy\x)S;^Np{x)Sp'] [{/5}(x)Bi + /iM(x)] mp+i(x) 
+ WphP^\fgr\x)S;\fg}{x)B2mp+2{x) + o{hP+') 
=hP+^WpSp^Biuip+i{x) + hP+^WpSp^ [{fg}~Hx)uip+i{x){Mix) - Np{x)Sp^Bi} 
+ 52mp+2(x)] +oihP+^). 

We claim that for elements EP*j.{x) of EP*{gi) with |r| even, the /i^"*"^ term will vanish. This 
means for any given r with \r\ < p and £2 with \r2\ = P + 1; 



o<lrl<p 



0. 



(26) 
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To prove this, first note that for any with < |r^| < p and £2 with \r2\ = P + 1) 

^ {Sp^}N(r^),N{r) ^r+r^ = j if^^r^A'^du, (27) 

0<|r|<p 

where Kr^p{u) = {\AIr^p{u)\/\Sp\}K{u) and Mr^p{u) is the same as Sp, but with the iV(r) column 
replaced by fJ,{u). Let Cij denote the cofactor of {Sp}ij and expand the determinant of Mr,p{u) 
along the N{r) column. We see that 

/ U-'^Kr^p{u)du= \Sp\~^ / C^(^j.),N{rj^)yf^~^~^{li)d'U- 

0<\r\<P 

([27|) thus follows, because C7v(r),7V(r j/l'S'pl = {Sp'^}N{r-^),N{r) from the symmetry of Sp and a 
standard result concerning cofactors. As a generalization of Lemma 4 in Fan et al (1995) to 
multivariate case, we can further show that for any with < |r ^ | < p and p — \ri\ even. 



J vf^ Kr^p{u)du = 0, for any jrgl = p + 1, 



which together with ([27|) yields to ([M])- □ 
We proceed to prove the main results Theorem l3.2[ Define = X_^—x, = u(Xf^), Ki^ = 
KhiKix) ^-iid (pni{x;t) = (p{Yi; fil^Pp{x) + t). For a, /3 G TZ^ , define 

$„i(x; a, /3) = Kix\^piYi; fi^^ia + p + /3p(x))) - p{Yi; iil^iP + Ppix))) - ipi{x; 0)//y^a} 
= Kii: / {ipniix;t) - ipni{x;0)}dt, 

and -R„i(x; a, f3) = ^niix; a, (3) - E^niix; a, (3). 

Lemma 5.1 Under assumptions {Al) — {AQ), we have for all large M > 0, 

n 

sup sup I Rnijx; a, (3)\ < M^^'^dn almost surely, (28) 
where B^i^ = {/? G : \HnP\ < M^'^}, i = 1,2. 
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Proof. Since P is compact, it can be covered by a finite number of cubes T>k = T>n,k with 
side length /„, = 0{Tn^^^) = 0{/i(n/i'^/log n)^*^^^'*'^)/^} and centers = x„ Write 

n n 

sup sup \y^Rni{x;a,(3)\< max sup \y^^ni{xk;a, f3) - E^niixk, a, P) 



n 

+ max sup sup ] ^nifefc; a, /?) - ^mfe a, /3) \ 

n 

+ max sup sup \ E^ni{x,^; a, P) - E^ni{x; a, (3) \ 

+ Q2 + Q2 



/3 e 



In Lemma E21 it is shown that Q2 < M^/'^dn/^ almost surely and thus Q3 < M^^'^dn/S. 

(i) 

Now all we need to do is to quantify Qi. To this end, we partition Bn , i = 1,2, into a 

(i) (i) 

sequence of disjoint subrectangles D\ ■ ■ , D such that 

\Df^\ = sup [\Hr,{a -p)\:a,p£ Z)jf } < 2M-iM«/logn, 1 < ji < Ji. 
Obviously Ji < (Mlogn)^. Choose a point a^j G Dj^^ and € -D^^^. Then 

n 

Qi < max sup | V'l^ml^fc; "jd /^fci) - -Rnifefc; 

n 

+ max I Rni{x^; aj^ , /JfcJI = -ffni + Hn2- (29) 

1 < A: < T„ 
1 < < Ji 

(i) 

We first consider Hni. For each ji = 1, • • • , Ji and i = 1,2, partition each rectangle Dj_^ 

(i) (i) 

further into a sequence of subrectangles Dj_^\,-- - ,D-_^j^. Repeat this process recursively as 
follows. Suppose after the /th round, we get a sequence of rectangles D) ■ ... • with 1 < jfc < 
>/fc) 1 < < ^, then in the (Z + l)th round, each rectangle D^j^ ... is partitioned into a sequence 
of subrectangles {D^f^ ■ ... ■ ■ , 1 < ji < J;} such that 

l^]:!..-...,.J = sup{|i^„(a-/3)| : a,/3 G Z^JJ^ J < 2M<^)/(M' log n), 1 < < J,+„ 
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where J;+i < M . End this process after the (L„ + l)th round, with L„ given at the beginning 
of Section 3. Let D^i \ i = 1,2, denote the set of all subrectangles of Dg*^ after the Ith. round of 

(i) (i) (i) (I) 

partition and a typical element Dj^-^ ... of is denoted as Dy^y Choose a point a(jj) € D^^j^^ 

(2) 

and € -^(jj) and define 

<3/ = y'-Pi sup y'{i?m(^fc;aip - -Rnifefc;",/?)} > k 1 < ^ < Ln + 1- 

By (A4), it is easy to see that for any a € -D^^ G -^lI+i ^'^'^ ^ '^(^l +i) ^ -^lI+I' 

CMn^ 

which together with the choice of L„ implies that Ql„+i = 0. As < + Q/, 1 < / < L„, 

/i#3/2j L„ 

> ^-^) < T„Qi < T„ J] V^. (30) 

To quantify VJ, let 

n 

Wn = ^^Zni, Zni = Rni{x^;aji,(3k,) - Rni{Xk;aj^^-^,Pji^^). (31) 
i=l 

Note that by (A2), we have, uniformly in x, a and /?, that 

P)\<CMi'\ (32) 
Therefore, \Zni\ < CM^^\ With Lemma EH we can apply Lemma 15.41 to Vi with 

r„ = ri = (2^^2/2/^)''/"'^"), 9 = n/rl ri = M^/^d^/l', 
A„ = (2CiM(i)ri)-i, ^'(n) = Cq'/' /v^HrlRrM'^V^^ 

Note that nMn^ /rj ^ oo, ^ cx) for all 1 < / < L„ from ([19]) and 

Ar? = iQg^^^2iM/22', A^Ss = Clog7i^-2/^w2'/^V2^' = o{\ri), 
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which hold uniformly for all 1 < / < L„. Therefore, 

l+i 

Vi < (n«/|)4exp{-Cilogn(M/2'^2)2V.2} + c'2T^, 
i=i 

where, as Ji < 2(Af logn)^ and J/ < 2M^ for 2 < / < L„, is given by 

n. 



tI = 4'M2^('+^)(logn)2^n3/2 2M|lg^^ 



It is tedious but easy to check that for M large enough, 
L„ l+l 

T^nJ2[{]l >^|)4exp{-Ci logn(M/2''2)2Vi'2} gummable over n. (33) 
1=1 j=i 

As Ti?"^]/?"^ is increasing in /, we have 

TnEr^ < T„(logn)^^n3/^ ^^"'^|'f ^^^^^^ n4^M^^('+^), 

which is again summable over n according to (j2ip . This along with (j30p and (j33p implies that 
-f^ni ^ almost surely, by the Borel-Cantelli lemma. 

For Hn2, first note that 



P{Hn2>v) < TnJfP{\J2Rni{x;aj„pk,)\>ri)- (34) 

i=l 

We apply Lemma [57i] to quantify P(| Y17=i Pni{x', Ojn Ai I > ??), with r„ = r(n), Si = 2CiAIn\ 
B2 = C2nh'^{M!i^^ fM^\ \n = {r{n)M^^y^/ACi and = M^l'^dn. Then n^i/r/ ^ 00 and 

A„r?/4 = (n/i^)(^-^2)/2(^Qg^)(i+A2)/2^|-^g^^^^^^| ^ ^1/2 iog„/(i6C7^)^ 

A^^a = M^I\nh'^f-^^{\ognf^/{lQClr'^{n)} = M^'^ logn/(16C72), 
*(n) ^ g„{nBi/r?}i/2^[rJ = T„j2g(n)3/Vr?V2^[r(n)]{r(n)M(i)}V2, 

where \I'(n) is summable over n by condition (|2ip . Therefore, 

P(^^n2 > r?) < 2T„,j2/n^ + ^(n), 6 = t7^(mV2 _ M^^^). (35) 

By selecting M large enough, we can ensure that (I35p is summable. Thus, for M large enough, 
-f^n2 ^ M'^l'^dn almost surely. By (|57p . we know for large M, Qi < M^/'^dn almost surely. □ 
The quantification of Q2 is very involved, so we put it as a separate Lemma. 
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Lemma 5.2 Under the conditions in Lenima \5.1[ Q2 < M^^'^dn/3 almost surely. 

Proof. Let 2[.ik = K.i ~ ~!Lki l^ik = uiX^f.) and Kn^ = Kh(X^f^). It is easy to see that we can 
write ^ni{xk] a, /?) - ^ni{x; a, /3) = in + Ci2 + Cis, where 

: {^ni{Xk; fjJkif^ + at)) - ifniiXk, 0)| dt, 

ii2 = Kic^fil^a jy^mfeA.-; tikiP + a*)) - "fmix] ii[^{l3 + tit, 
Ci3 = -ftrix^Txa{y'ra(2;; 0) - Lpniix^; 0)}. 
Then P(Q2 > M^l^dn/^) < T„(P„i + P„2 + Pm), where 



Cii = ( KikHik — Kix^iix ) a 



Pnj= max P( sup sup | V ^^^1 > M^/^d^/g) , j = 1, 2, 3. 
i<fc<T„ ^xe7?fe„gs(i)^ ^ ^ 

Based on Borel-CanteUi lemma, Q2 < M^/'^dn almost surely, if X]„T„P„j < 00, j = 1,2,3. 

We first tudy For any fixed a E Bn^ and /3 € Bn \ let /^"^'^ = 1, if there exists some 
t S [0, 1], such that there are discontinuity points of ip{Yi; 0) between lJ^j^{Pp{xk) + P + at)) and 
/^Tfc/9p(^fc); and If/ = 0, otherwise. Write ^ii = ^ii/f/ + - ^f/)- Note that by (A3), 
\{KikHik-Kixfiix)^a\ < C2Mr^Pln/h. Then by (A2) and the fact that |^j^(/3 + Qt)| < Cil4^\ we 
have |^ii(l - < CMi^'^Mi^h„/h uniformly in i,a, [5 and x G V^. Define C/jfc = I{\Xik\ < 

2/1}, whence .^ji = S,iiUik since Z„ = o{h). Therefore, 

M3/2d„\ MVW^ 



/3 e Bk'^ 



18C 



< P(|^^.fc-g^.fc|> ),(36) 

where the second inequality follows from the fact that Var(^"^-^ < 2/i) = 0{nh'^) implied 

by Lemma lS.Sl To quantify ()36p . we apply Lemma [5.41 with Bi = 1, rj = M^^^nh'^ / {18C) , B2 = 
nh'^, rn = r{n). As An?? = CMV4 log log n)(i+^2)/2^ ^2^^^ ^ o(A„r/) and T^^-^ is 
summable over n under condition (I21D. we know that 



n 

T„pf sup |Veii(l-/f/) > Af^/2(i„/18) is summable overn, (37) 
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whence X^„T„P„i < oo, is equivalent to 

n 

T„P(^ sup I ^Ca^^'^ > M^^^d„/18) is summable overn. (38) 

aeBi^\ i=l 



IS e Bi?^ 



To prove §8^, first note that /f^''' < I{ei £ }, where 

i=ltG[0,l] 
m 

, Oj + CM^^)] = Dn, for some C > 0, 

i=i 

1 /"^ 

-4(xi, = (p + 1) V -rfe - X2)- / D-m{x2 + - X2))(l - wfdw, 

|r|=p+l 

where in the derivation of 5*"^^ C Dn, we have used the fact that \2[.ik\ ^ 2/i and A(X^,x^) = 
0{hP+^) = 0{a4^'^) uniformly in i. As If/ < I{ei G we have < \C^l\Unu where 

Uni = Ii\2Lik\ ^ '^h)I{ei G Dn}, which is independent of the choice of a and /?. Therefore, 

n n 

P{ sup I > M'/^dn/ls) < p[Y.Um > M'/^nh^Mi^^/{18C) 



'a e ' i=l 

/3 e Bi^) 



< - > l^^^'- ), (39) 

1=1 

where the first inequality is because < CMn^ln/h and the second one is because EUni = 
0{h'^MP) by (Al). As EU!^- = EUni, by Lemma[531 we know that Vai{J27=i Uni) = Cnh'^MP. 
We can then apply Lemma 15.41 to the last term in ()39p with 

B2 = Cnh'^Mj^^\ Bi = 1, r„ = r(n), r? = Mi/2^/i^iV42)/(3g(-.)^ 

Apparently, A„7/ = Clogn(n/i'^/logn)(^~''*2)/^ and A^-B2 = o{XnTi). As in this case T„^'„ is still 
summable over n by (j2ip . (j38p thus follows. 

For Pn2, first note that using approach for P„i, we can show that 



T„P( sup sup y^{^j2 — C«2} > M^^^dn/ 18) is summable over n. 



/3 e Bi?^ 
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where 

"1 



li2 = Kikl4ka J |(/?„i(xfc; /i[fc(/3 + at)) - ipni{x; fiJ^iP + at))| dt. 
Therefore, we would have ^ T„,P„2 < oo, if 

n 

TjiPy sup sup y^C«2 > M^^'^dn/is) is summable over n. (40) 

For any fixed a € /3 G i?^^^ and x € Pfc, let -/^j"^''^^. = 1, if there exists some interval 

[ti,t2] C [0, 1], such that 

y^ - f^lkWpixk) + P + at)< aj <Yi- /iL(/3p(x) + /? + at), Vi € [ti, ts] (41) 

with G {ai, ■ ■ ■ ,0^}; and = 0, otherwise. Write fj2 = 6!2^°fc'^a; + |j2(l - -^"^fa;)- Note that 
Kikf^lk'^ = 0{M^n^) and v?„i(x;,; ^j,(/3 + at)) - </;™(x;^L(/3 + at)) = 0(Mf )/„//i) if I'^f^^ = 0. 
Then again as = Ci2-^{|:^i/fc| < 2/i}, we have similar to ([37|) that 

n 

T„P(^ sup \'^ii2{l - Itf,x) > M^/^dn/ls) is summable overn. 

a(^B^^\ i=l 

Therefore, by (j40p . to show ^ T„P„2 < cxo, it is sufficient to show that 



T„P( sup sup \ ^ii2ltix ^ M^/^dn/Se) is summable over n. (42) 



To this end, define ei = £i + A{X_^,x^). Then = 1, i.e. (jlT]) is equivalent to 

>l(^i,^fc) - ^(^i,ic) + ^L(/3 + at) <ei- aj < + at), Vt € [ti,t2]. (43) 

Let 6n = Mi^hn/h. Then |A(X„rE;..) - ^(X„x)| < C6n, K^ife - /Xi,)^/5| < C6n and (gSD thus 
implies that 

-2C5n + fiJkiP + oit) <^i-aj < fiJk{P + oit) + 2C5n, Vte[ti,t2]. (44) 
Without loss of generality, assume A^fcO > 0. Then from ([3H) we can see that 

- 2C5n + fiJkiP + at2)<ei- aj < fiJ^iP + ah) + 2C75„, (45) 
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which in turn means that if = 1, then |^j2| < C{t2 — ti) | < 4C5„ uniformly in 

i, a € B^n \ (3 € Bi^^ and x £ Vk- Therefore, as = ii2l{\X-,J\ < 2/i}, we have 



n 

P( sup sup J^|i2/"£, 

/3 6 



> 



36 



^gfiW-e^?,^ 366 y 

We will bound I'j^f^ by a random variable that is independent of the choice of a G Bn^ and 
X € Dfc. By the definition of and (SSI), the necessary condition for I^f^ = 1 is 

m 

G U [a, + - 2M(i) , a, + + 2M(^)] Df„ (47) 
which is indeed independent of the choice of a and x G P^. Therefore, 

P( sup sup 2^/{|Xifc| < 2h}I^.^ > — 

(2) 

Now we partition Bn into a sequence of subrectangles 5i, • • • , 5m, such that 



\Si\ = sup - : G 5;} < mW, i<i< 



m. 



Obviously, m < (Af^^V^^n^^)^ = M~^^/'^{nh'^ / logn)^^^'^^^^ . Choose a point A € 5; for each 
1 < I < m, and thus 

1=1 

+mP( sup X:n|X.,| < 2/.}|/{e. G " ^{^. ^ > "^'^'t'"^"'^ 

= m{Ti+T2). (49) 
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We deal with Ti first. Let 

Ul^I{\X,^\<2h]I{ei&D^^,}. (50) 

Then by the definition of D^l given in (gT]), EU^^ = OQi'^mI^^) < M^/^h'^Ml^'^ /{UAC) for large 
M and we have 

i=l 

We can thus apply Lemma 15.41 to the quantity on the right hand side with Bi = 1, B2 given by 
(f69l) . r„ = r(n) and r/ oc M^/^nh'^M!;t\ and A„ = l/(2r„). It follows that 

XnT] = CM^'^ log n{nh'^/ log n)(i+^2)/2-Ai ^ ^2 ^ ^ ?i(n/i'^/ log n)^2{Ai-A2)/i^2 _ 

As (1 + A2)/2 > Ai and A2 < Ai, we have Ti = 0(n-^) for any 6 > 0. 

For T2, note that as |^J;j(/9 — A) I ^ CMn^ for any /? G S*;, 1 < / < m, we have 



■m 

for some C > 0, which is independent of the choice of /3 G S"/. Therefore, 



T2<PQ]/{|X,,|<2MC/n.> 



72C 



which can be dealt with similarly as with Ti and thus T2 = 0{n~ ) for any 5 > 0. Thus from 
(j46p . (j48p and (j49p . we can claim that (j42p is true and thus T„P„2 is summable over n. 

The quantification of P„3 is much simpler, as no /3 is involved in ^43. For any given x G I'fc, 
let /i;fc,a: = 1, if there is a discontinuity point of ip{Yi;6) between fili^(3p{xf^) and fjJxf^pi2L)'i 

= otherwise. Write S,i3 = i^Ii-k^x + Cj3(l - Ii;k,x)- Again by (A2) and the fact that 



|ir„/iLa| = 0(m(^)) and \li\l3,{x^) - IxJMx)\ = \A{X^,x^ - A{X,,x)\ = 0{Mi'hjh) 
have similar to ([57P that 

n 

T„P( sup y^Ci3(l ~ Ii;k,x) > M^/^(i„/18) is summable over n. 



we 



x&Vk 
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It's easy to see that Ii-k,x < + A[K.i^Xf^) ^ Si-k,x}^ where 

m. 

Si-Ax = U U [^3-\MX^,x^)-A{X^,x)\,a, + \A{X,,x^)-A{X,,x)\ 

j=lfg[0,l] 



m 



C IJ [aj - CMl^hn/K aj + CMPln/h] = Dn, for some C > 0. 

Therefore, \Ci3\Ii;k,x = l^islHlKikl < '^h}Ii.k,x < Uni, where 

Uni = M(i)/{|X,fc| < 2h}I{e, + A{X^,Xk) G D^}, 
which is independent of the choice of a G Bn^ and x £ "D^. Thus 

n n 

T„P( sup I ^^3I^■,^ > M^/^d^/ls) < T„p( ^[[/^ - EUni] > M^/^d^/So) , (51) 

a e bI^^ i=l i=l 

where we have used the fact that EUni = 0{h'^Mn^ Mn^ln/h) = 0{dn/n). We will have 
T„P„3 < oo if the right hand side in ([5T]) is summable over n, i.e. 

n 

T„P(^ - £;C/„i] > M^/^dn/Se) is summable over n. (52) 

1=1 

It's easy to check that Lemma [5 . 5 1 again holds with il^xiX^^Yi^ standing for Uni- Applying Lemma 
Oto ([52]) with Bi = Mi^\ B2 = Cnh'^{Mi^^fMPln/h, r] = M^'^dn/^Q and r„ = r(n), we 
have (note that nBi/rj — > 00 indeed) 

Xnri/A = CM^/^ logn, A^Ps = Cr'^/^^ i^g^ ^ ^^^nv)- 



Thus, T„^'„, again is summable over n and (j52p indeed holds. □ 
Proof of Theorem 13.21 Let Ai = A(s). Then according to Lemma l5.ll and Lemma 15.91 we 
know that with probability 1, there exists some Ci > 1, such that for all large M > 0, 

sup sup ^ni{x; a, (3) —{Hna)^ Snp{x)Hn{a + 2(3) 

< CiM^''^{dnl + dn) < 2CiM3/2(n/l'^)l-2Ai(lQg^)2Ai^ ^^^^ -g ^^^gg^ (53) 

where dni = {nh'^)^~^^~'^^'^ (logn)'*'^'^'^^'^ . Note that from p2]) . we can write 

n 

^i^'„iV3(yj;/i^j/?p(x))/i]^jQ; = n/i'^/5*(x)^VFp~^S'„p(x)if„a. 



1=1 
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Replace B^it^ in ([53]) with B^^l = |a G : A; < M~i(n/i'^/ log n)^i |if„a| < A; + 1} and M with 
{k + 1)M. We have, by the definition of $nj(^; «, that 

n n 

/3 e 



> inf inf — (F^af 5„p(x)i/„a-2CM3/2(n/i^)i-2Ai(^Qg^^2Ai 

> |C3(fcM)V2-2(:7i(/fc + l)3/2Af3/2|(j^/irf)l-2Ai(iQg^)2Ai 

> (8 - 2^/^)CiCl^^{nh'^)^-^^'{lognf^^ > almost surely, (54) 

where the last term is independent of the choice of A; > 1. The last inequality is derived as follows. 
As Sp > 0, suppose its minimum eigenvalue is ri > 0. As 5'„,p(x) g{x)f{x)Sp uniformly in 
X ^ T) hj Lemma 15.81 and g{x)f{x) is bounded away from zero by (A5) and (j24p . there exists 
some constant C3 > 0, such that for all x G P, the minimum eigenvalue of S„p(x) is greater 
than C3. The last inequality thus holds if M > C4 = (16Ci/C3)2. Note that 

U B« = {a\ , n- : {i^f'\H,.M > m} := (55) 

fc = l 

Therefore, from (1541) and (I55D. we have 



ri n 

+nh'^{Wp^l5l{x) - Hnl3f Snp{x)Hna^ > almost surely. (56) 

Note that by (j58p . Lemma 15.101 and Proposition 13. H we have |/3^(x)| < C 3 {nh'^ / log n)~^'^ 
uniformly in x^ V almost surely. Namely, /3j^(x) S Bn^ for all x € I^, if M > C|. This implies 
that if M > max(C|,C4), §Ej still holds with f3 replaced with H-^W'^ (3*{x). Therefore, 



At 

K I E KmP{Y^;pUa + H~^W;^(il{x) + /3p(x))) 

n 

Y,KmP{Yv,p'^,[H-^W;^Pl{x) + Pp{x)))] > 0, 



" 1=1 
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which is equivalent to Theorem 13.21 □ 
Proof of (I13p . Let d„ = (n/i'^)^^^^^ (log n)^^^ . Through the proof lines of Theorem 13.21 we can 
see that (|13|) will follow if 

n 

sup sup I Rnijx', Qi P)\ < M^/'^dn almost surely, 
with Ai = 1, A2 = 1/2 and B^^\ i = 1,2 defined as in Lemma |5. 11 

To prove this, cover D by a finite number T„ = {(n/i'^/ log n)^/^//i}'^ of cubes = T^nk with 
side length /.„ = 0{/i(?i/i'^/ log n)~^/^} and centers Write 

n n 

sup sup I V]-Rm(^;a,/3)| < max sup V] a, /3) - ^$„j(^fc; a, /3) 



n 

max sup sup \ ^ni{xj^; a, /?) - ^'nife a, /?) [ 



/3 e Bi^^ 



71 

+ max sup sup > < £'<I'„j(x^; a, /3) — i?$rii(x; a, /5) > 



/3 6 Si^' 



We will show that with probability I, Qk < M^/'^dn/^, A; = 1, 2, 3. 

Define iij as in Lemma [SH As P{Q2 > M'^/'^dn/2) < t„(P„i + P„2 + -Pns), where 

n 

Pnj= max P( sup sup > M^^^dn/d) , j = 1,2,3. 

l<fc<T„ ^3l67?fc„gB(l)^ ^ 

Then based on Borel-Cantelli lemma, Q2 < M^^'^dn/2 almost surely if X]„T„P„j < 00, for 
j = 1,2,3. We only prove that for P„i to illustrate. Recall that 



T f 

Cii = (^Kikl^ik - Ki^f^ix^ a j ^ifniixk, fJ^ikiP + at)) - ^ni{xk, 0)| 



dt. 



Because \{Kikfiik — Kixfiixja I < CiMi^^ln/h, IfiJf^iP + at)\ < CM^^ and ip{.) is Lipschitz 
continuous, we have < CMn^ Mn^ln/h. Define Uik = I{\2[.ik\ — ^^l- = 0(^)1 we can 
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see that = CnU-ik and similar to (j36p . we have 

P sup sup V^ii > ^ < P Vf/*fc> 



/3 e Si^^ 



9C 

i=l 

and ^„ TnPnj < co thus follows from similar arguments as those lying between (j36p and ([3j 
The proof of Qi < M^/^(i„/2 almost surely is much easier than in Lemma [5 .11 is Lipschitz 

continuous. Instead of the iterative partition approach adopted there, we once for all partition 
i = 1,2, into a sequence of disjoint subrectangles d\^\ • • • , Dj^ such that 

I = sup [\Hn{a - : a,/3 G D]f } < M« (log n/n)V2, 1 < < J,. 
Obviously Ji < (n/logn)^/^. Choose a point G D^J^^ and (3ki S -^i^^- Then 

n 

Qi < max sup | V'l-Rmfe; "jd /^/cj - -Rnife; a, 

n 

+ max I Y] Rni {x^ ; aj, , ) | = + Hn2 ■ (57) 

1 < fc < T„ ^ 
1 <ii,fci < Ji 

By Lipschitz continuity of (/?(.), we have for any a € -Dj|^^ and /? G 

|^m(^fc;«ii, - ^™(^fc;a,/3)P = 0({M(2)}3 log n/n) < M3/2J„/(4n). 

Therefore, it remains to show that P{Hn2 > M'^^'^dn/^) is summable over n. 

First note that by Cauchy inequality, |i?„i(x; a, /3)p = 0{{M^^ M^^Y) and -B|i?„i(x; a, = 

0(/i'^{m1^^M^^^}2) uniformly in X^, x, a G M^,^^ and /? G M^^^. Next, for any r/ > 0, 

n 

P{Hn2 >ri) < TnJlP{\Y,Rni{x;aj,,f5k,)\ > r/). 

4 = 1 

We apply LemmaEaiwith r„ = (n/i"'/ log n)i/2, Si = 2CiM^^^Mf \ ^2 = C2n/i°'(M^^^MP)2, A„ 
(4Cir„{M^^^}2)~^ and = M'^/'^dn/^. It is easy to see that nBi/r] oo and 

A„r?/4 = Mlogn/(16Ci), A^^s = o(A„r?) 

^(n) = gra{ra-Bi/r?}^/^7[rri] = n^/^(log n)~^/^7[r(n)]/r(n). 

29 



As fnJi'^in) is summable over n by condition (j2ip . so is 

Proof of Corollary 13.31 As 1 + A2 > 2Ai, it's sufficient to prove that with probability 1, 

1 1 1 V— -\ r /lo£rn\ (i+-^2)/2-i 

1=1 

uniformly in x G D. As (/^(ei) = ip{Yi,m{Xi)) and Eip{ei) = 0, the term on the left hand side of 
(l58l) stands for 



1=1 

where 

Next, like what we did in Lemma l5.ll we cover T> with number T„ cubes Dk = T^n,k with side 
length In = 0{Tn^^'^) and centers x^ = x^^k- Write 

n n 

EZni{x) - EZni{x)\ < max Zni{x,^) - EZniiXf,) 

1 <i<-<T — 

n 

+ max sup Zni{x) - Zni{x^) 
i<k<T„xeVk ' ~l 



sup 



n 

+ max sup E^ni{x) - EZnii 
i<k<T„xeVk I ~^ 

=Qi + Q2 + Q3- 



As Zniix) - Zni{x^) = Hn^Kh{X_i - x)fi{Xi - x){^pn^ix■,0) - (Pni{xk]0)}, through approaches 
similar to that for in the proof of Lemma 15.21 we can show that 

i¥ \(i-A2)/2 



Q2 = 0< logn > almost surely 

I V loe nJ ) 



^ log n / 

and so is Q'^. To bound Qi, first note that EZ^-{xf^) = 0{hP~^^'^'^) uniformly in i and k. As 
\^ni{x)\ < C for some constant C by (A2), we can see that from Lemma 15.51 

n 

EZlixk) + J2 \Cov{Zm{x,), Zn,{xk))\ < C2nhP+'+^. 

i=l i<j 
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Finally by Lemma EH with Bi = Ci , B2 = CnhP~^^+'^, r] = ^3(71/1'^/ log n)(^~^2)/2 ^ ^^^^^ 
Tn = r{n), we have (note that nBi/rj —>■ 00 indeed) 

Xnv = As/{2Ci) logn, A^Ss = C2/{4C!) logn. 

Therefore, 

n 

P( max \y^Zni{xk)-EZni{x,,) > ylg^V log n)^^-^-)/^ logn) < T„K + CX„^„, 



i=l 



where a = As/iSCi) - C2/(4Cf). By selecting ^3 large enough, we can ensure that T„/n" is 
summable over n. As T„^'„ is summable over n from (j2ip . we can conclude that 

ih'^ \(l"A2)/2 



= logn } almost surely. 

I V loe n / ) 



. log n / 

This together with Lemma 15.81 completes the proof. □ 
Proof of Corollary 14. li Through the proof lines for Theorem 13.21 and Corollary 13. 3|. it's 
not difficult to see that Corollary 13.31 still holds under the conditions imposed here. Under the 
additive structure ([!]), we thus have 

<pni{xi) =(^i(xi) + - y^m,2(^2i) - hP+^eiWpSp'^Bi-'^nip+i{xi,X2i) 



n — ' n 

1=1 i=\ 



j=i i=\ 

+ Op({max(/ii,h)}P+i) + Op{(n/ii/i'^"Vlogn)-3/4}, (59) 



where = X\j — x, 2L2,ij = 2L2i ~ 2£.2j ™d ei is as in Proposition 13.11 Note that by (flTl 

(n/ii)^/^(n/ii/i'^~"'^/log n)~^/^ — > 0, the Op{.) term can thus be safely ignored. 

By central limit theorem for strongly mixing processes (Bosq, 1998, Theorem 1.7), we have 



-^rn2{X2i) = Op{n-^/^), - ^ mp+i(xi, = Emp+i{xi, X2) + Opin'^ 



/2l 



n ^ — ' n 

1=1 1=1 



As the expectations of all other terms in (j59p are 0, the leading term in the asymptotic bias of 
01 (xi) — 4>i{xi) is thus given by 

-{max{hi,h)r+^eiWpSp^BiEnip+i{xi,X2). 

31 



Again through standard arguments in Masry (1996), we can see that 
1 " 

^f^d-l ^np i^l > K2i)Kh {Xl,xj , 2L2,ij)K^l,xj I hi , X_2,ij/h) 
i=l 

uniformly in 1 < i < n. Therefore, the leading term in the asymptotic variance of 
is the variance of the following term 

n „ 

{nhi)~'^ei^ip{ej)S~p{xi,X2j)f2{X2j) / {K^t}{Xi^xj /hi,v)dv, 

which is asymptotically 

(n/ii)-i| / {f9^r\x,,X2)fl{X2)a\xi,X2)dxAeiS-'K2KlS;'e[. (60) 

J[o,i]'»''-i 

If p(y-0) = (2q - l){y -0) + \y - e\ and ip{e) = 2ql{e > 0} + {2q - 2)I{6 < 0}, we have 
9{x) = 2/,(0jx) and 

a\x) = E[^\e)\X = x] = V(l _ F,{0)) + 4(1 - q?FM = Aq{l - q), 

which when substituted into (j60p . yields the asymptotic variance for the quantile regression 
estimator. 



ixi)=q{l-q)\ [ r\xi,X2)f-H0\xi,X2)fliX2)dX2}eiS;'K2KlS;'e[. □ 



'[0,1]« 

The next Lemma is due to Davydov (Hall and Heyde (1980), Corollary A. 2). 

Lemma 5.3 Suppose that X and Y are random variables which are Q— and TC— measurable, 
respectively, and that E\X\p < oo, £'|y|'^ < oo, where p, q > 1, p"^ + q~^ < 1. Then 

\EXY - EXEY\<8\\X\\p\\Y\\g{ sup \P{AB) - P{A)P{B)\y''' "'^ . 

A&g,B&H ^ 

The next lemma is a generalization of some results in the proof of Theorem 2 in Masry (1996). 
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Lemma 5.4 Suppose {Zi}'^^ is a zero-mean strictly stationary processes with strongly mixing 
coefficient 'y[k], and that \Zi\ < Bi, EZf + |Cov(Zj, Zj)\ < B2. Then for any 77 > 

and integer series rn — > 00, ifnBi/r] — > 00 and Qn = [n/r„] 00, we have 



where ^{n) = g„{nBi/??}V2^[r„], A„ = l/{2r„Si}. 

Proof. We partition the set {1, • • • ,n} into 2q = 2qn consecutive blocks of size r = rn with 
n = 2qr + v and < i; < r. Write 

jr 

Vnij)= Yl Zi, j = !,■■■ ,2q 

j=(j-l)r+l 

and 

y q n 

W'^ = Y, Vn{2j - 1), < = 5] K(2j), Wli' = 

j=l j=l i=2(j'r+l 

Then Ty„ = ^"^^ = + 1^;^' + Wli'. The contribution of VF;;" is neghgible as it consists of 
at most r terms compared of qr terms in or Wl[. Then by the stationarity of the processes, 
for any ?? > 0, 

P{Wn >v)< P{W:, > 77/2) + P«' > r//2) = 2P« > 77/2). (61) 

To bound P{W^ > rj/2), using recursively Bradley's Lemma, we can approximate the random 
variables Vn{l), V„(3), • • • , Vn{2q—1) by independent random variables V*{1), V*{3), • ■ ■ , V*(2q— 
1), which satisfy that for 1 < j < q, V*{2j — 1) has the same distribution as Vn{2j — 1) and 

P(\v:{2j -1)-Vn{2j -l)\>u) < 18{\\Vn{2j - l)\\^/uy/^ snp\P{AB) - P{A)P{B)\, (62) 



where u is any positive value such that < u < \\Vn{2j — l)||oo < co and the supremum is taken 
over all sets of A and B in the o"— algebras of events generated by {y„(l), V„(3), ■ ■ • , Vn{2j — 
3)} and Vn{2j — 1) respectively. By the definition of Vn{j), we can see that sup\P{AB) — 
P{A)P{B)\ = 7[r„]. Write 

1 g 



P{W'^>i) <P j;y;(2j-l) >l\+p(\YVn{2j-l)-V:{2j-l 



4. 



>4 



2- 

= h + h. (63) 
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We bound Ii as follows. Let A = l/{2Bir}. Since \Zi\ < Bi, \\Vn{j)\ < 1/2, then using the fact 
that < 1 + X + holds for |x| < 1/2, we have 



{2j-i)}2 



By Markov inequality, ([M|) and the independence of the {y^(2j — 1)}^^^, we have 



h < e-^^/^\Eexp(x^V:{2j-l))+Eexp(-X^V:i2j-l 



< 2exp(-A7?/4 + A2J]i?{V;*(2i-l)}^ 



< 2exp{ - A??/4 + C2A2S2}• 



We now bound the term I2 in (j63|) . Notice that 



/2<5]p(|K(2j-i)-K:(2i-i: 



> 



Aq 



If ||K(2j - l)||oo > f?/(4g), substitute r//(4g) for u in §2 



h < im\Vn{2j - 1)||/W(49)}^/^K] < C7g3/2/r?V2^[r-„](r„Si)V2, 



If \\Vn{2j - l)||oo < ??/(4g), let u = \\Vn{2j - l)||oo in ([62]) and we have 



h < Cqj[rn], 



which is of smaller order than (|66]) . if nBi/i] 00. Thus by (i6T]) . (f63l) . ([65]) and ([Ml 



P(M^n > r?) < 4exp{-A„r//4 + C2B2XI} + C$„, 



(64) 



(65) 



(66) 



where the constant C is independent of n. 



□ 



Lemma 5.5 For any x € R'^, let il^r.(X^,Yi) = I{\2Lix\ — h)il)x(Xf^, Yj), a measurable function 
of (Xf,Yi) with l-i/^^fXj, 1^;)| < B and V = E'4!^{X_^,Yi) . Suppose the mixing coefficient '^[k] 
satisfies (2D(l . Then 

Cov(f; |V'.(x„ y,)|) = nv [1 + o{ {b^h^^^^^/v) } 

i=l 
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Proof. Denote ipxi^i,Yi) by V'ix- First note that 

V = Eipf^ = h'^ E{ipf^ \Xi = x + hu)f{x + hu)du, 

J\u\<l 

n—d n—d 

^ |Cov(V'jx,V'ix)| = '^{n-l-d+l)\Cov{'tpox,'ilJix)\<n^\Cov{Tpox,i^ix)\ 

i<j 1=1 1=1 

d—l TTn n — d 

= n^+n^+n ^ = n J21 + n J22 + ^^J23, 

1=1 l=d i=7r„ + l 

where vr^ = /i(P+''+i)(2/''2-i)/a^ Yot J21, there might be an overlap between the components of 
Xq and 2Lh foi^ example, when 2Li = {^i-d, ■ " " ^Xi^i), where {Xi} is a univariate time series. 
Without loss of generality, let v/jvf' and vf" of dimensions l,d — I and / respectively, be the 
d + l distinct random variables in (XQ^/h, Xj^Jh). Write = {vT ,vf'^y and U2 = (l* > 1*'"^ )^ • 
Then by Cauchy inequality, we have 

1 /2 

E{i^Ox,^Pix\f^Z^ttl)\ < {E{i^l\Xo = x + hu,)E{^p%\X^=x + hu2)} = V/h'' (67) 
and through a transformation of variables, we have 

|Cov(-i/'02;, V'ix)! < /i'W \ f{x + hui,x + hu2]l)-fix + hui)f{x + hu2]l + d-l)\du'dv/^ 

l«2l<l 

where by (A4) and (A5), the integral is bounded. Therefore, 

d-l 

nJ2i < CnVj2^^ = o{nV). 
1=1 

For J22, there is no overlap between the components of Xq and X_i. Let X_q^ = hu and 2Lix — 
and we have 

\Cov{^OxM\ < h'''' L^^E(^i^ox,i^ix\f^Zitt)'^^dv 

\v\ < 1 

x[/(^ + hu,x + hv;l + d — 1) — f{x + hu)f{x + hv)] 

= Ch'^V, 

where the last equality follows from (A4), (A5) and ([67|) . Therefore, as vr^/i'^ 0, 

nJ22 = 0{mrnh'^V} = o{nV). 
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For J23, using Davydov's lemma (Lemma 15. 3p we have 

|Cov(V'ox,V'/x)| < 8{7[/ - d + l]}^-2/"^^IV'.xrn'/"^ as V2 > 2. (68) 
As \i^^^\<B, E\^m\'^ <B^^-^V, 

00 

where the summation term is o(l) as 7r„ 00. Thus J23 = 0<^ y rB2;jP+d+l/y I ^j^j^^j^ 

completes the proof. □ 

Lemma 5.6 Suppose (A2)- (A6) hold. Then for U!^j^,l = !,•■■ ,m defined in iJQj) and Z^i^l = 
1, • • • , L„ defined in \31]) . we have 

n 

mL? + E C/^,)l < Cn/i'^M(i){M(2)/M(i)}i-2M, (69) 

j=l i<j 
n 

E ^^n. + E |Cov(^„., Z„,)| = n/i'^(M(i))2M(2){M' logn}-2/-^ (70) 
j=i j<j 

uniformly in Xf^, 1 < k < T„. 

Proof. We only prove (I70p . which is more involved than ()69p . To simplify the notations, denote 
Oj, ,/3fc, and by ai,f3i,a2 and /32, respectively. Clearly, 

{(Pni{x^;t)-ipni{x^;0)}dt = / 



rv^ H„ia2+I32) M H„{a2+l3i) 

/ {</2„i(xfc;t)-(/?„i(xfc;0)}dt = / {v9„i(xfc; t+27i/„(/32-/3i))-¥'m(^fc; 0)}dt, 



and 



i-u' Hn(ai+pi) rw Un(a2+P2) 

Zni = / Wni{Xk]t) - ipni{xi,;0)}dt - {^niix^lt) - ipni{xi,;0)}dt 

■JyJH„f3i JyJHul32 

= / Wni{xi^;t) - (fniiXk, t + Hn{(32 - Pl))}dt 

ru^ H„{a2+(3i) 

- / Wni{Xk;t + Hn{p2 - Pi)) - ^niiXk, 0)}dt = Ai + A2. 
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Therefore, ^{Z^J^ = h'^ f K'^{u)f{xk + hu)E{{Ai + A2f\Xi = + hu}du. The conclusion is 
thus obvious observing that by Cauchy inequahty and ([22|) . 



n) 



EiAl\X, = Xk + hu) < \]/ Hnai]/ Hnip2 - l3i)^Hnai \ < 2{M^^^fMP/{M^ logn), 
E{Al\Xi = Xfc + hu) < {y[Hn{a2 - ai)Y{\y^ Hna2 \ + \]7 Hnai\ + 2\u[ Hn(32\) 

< 4(MW)2M(2)/(M'logn)2, 

where we used the facts that \ai - a2\ < 2Mi^^ /{Aflogn) and - (32\ < 2MP/(Af'log 
Therefore, E{Zni}'^ = C/i^(M^^^)2mP/(M' log n). As < CM^^^ and hP+^/M^ < oo, the 
rest of the proof can be completed following the proof of Lemma I5.5[ □ 

Lemma 5.7 Suppose (A2)- (A6) hold. 

n 

Y,E^l, + Y,\GoY{^m,^nj)\<Cnh\M!^^^fM!i\ (71) 

i=l i<j 

uniformly in x G "D, a G Bn^ and /3 € Bn^ . 
Proof. By Cauchy inequality and ()22p . we have 



= /i"' / K\u)E \ / U'mfei) - V'rafeO) ^ |^i=:^ + /ili f{x + hu)du 

</i / fix + hu)K\u)ijL{uy Hna / S (^^(x; t) - 99„i(x; 0) |Xi = x + /iu 

<h'^ / K^{u)fi{u)^Hna / C|t|dt/(x + = h'^{M^}^fM^^ k (72) 

uniformly in ^ G P, a G i?!^"* and (3 G -Bi^^. ([TT]) thus follows from ([72]) and Lemma [531 □ 



Lemma 5.8 Lei (^3) - {A6) hold. Then 

sup \Snp{x) — g{x)f{x)Sp\ = 0{h + (n/i'^/ log n)~^/^) almost surely. 

Proof. The result is almost the same as Theorem 2 in Masry (1996). Especailly if (j2ip holds, 
then the requirement (3.8a) there on the mixing coefficient 7[/c] is met. □ 
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Lemma 5.9 Denote d^i = {nh'')^~^^~^^^{logn)^^+^^^ and let Ai and i = 1,2, be 



as m 



Lemma [5A[ Suppose that (Al) — {A5) and [19\) hold. Then there is a constant C > such that 
for each M > and all large n, 

sup sup I V E<^ni{x; a, /3) - "^{Hna)^ Snpix)H4a + 2/3)| < CM^/^dni. 

-GT^agfiW, i=l ^ 

Proof. Recall that G{t,u) = E{ip{Y ■,t)\X_ = u), 

r fixiuY H„{a+P) 

E^ni{x-a,l3) = h"^ K{u)f{x + hu)dux (73) 

+ lJi{uf Hnf3p{x),x + hu) - G{ii{uf Hnl3p{x),x + hu}^dt. 

By {A2>) and (^5), we have 

G{t + fJ.{u)^ Hn(3p{x) , X + hu) — G{fi{uj^ Hn(3p{x) , X + hu) 



f-2 

= tGi{fj,{u)~^ HnPpix),x + hu) + —G2iCn{t,u;x),x + hu), 
Gi{n{u)^ HnPp{x),x + hu) = g{x + hu) + 0{hP^^), 

where Cn{t,u]x) falls between ^^{uj^ Hnl3p{x) and t + fJ.{uj^ HnPp{x) , and the term 0(/i*'+^) is 
uniform in x G T>. Therefore, the inner integral in ()73p is given by 

^g{x + hu){Hna)''fi{u)fi{u)''Hn{a + 2/?) + ojikf^/^ (^i^)^^^'^'} 

uniformly in x€ T>, where we have used the fact that nh^~^^P^^^^^^ / log n < oo. By the definition 
of Snp{x), the proof is thus completed. □ 

Lemma 5.10 Under conditions in Theorem\3.SX we have 



sup 



^WpS,^p{x)H^ ^ X] -^^(^i ~ x)Lp{ei)^{X_i -x) = 0| (^^|^) ' } almost surely. 



1=1 



Proof. Note that, under conditions Theorem 13.21 the assumptions imposed by Masry (1996) 
in Theorem 5 are validated. Specifically, (4.5) there follows from (jl9p and (4.7b) there can be 
derived from (I21|) . Therefore, following the proof lines there, we can show that 

Li^-if;i^,(X, -x)^(s.)M^, -x)| = o{(i^)'^'}. 



sup 



n.. 
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which together with Lemma 15.81 yields the desired results. 



□ 
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